Pandas Profiling Report

Dataset statistics

Number of variables	11
Number of observations	6370
Missing cells	14644
Missing cells (%)	20.9%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	504.0 KiB
Average record size in memory	81.0 B

Variable types

Numeric	9
Categorical	1
Boolean	1

Alerts

`Date` has a high cardinality: 182 distinct values	High cardinality
`discount_promotional` is highly correlated with `discount_competitive`	High correlation
`discount_damaged_good` is highly correlated with `IsHoliday`	High correlation
`discount_competitive` is highly correlated with `discount_promotional`	High correlation
`IsHoliday` is highly correlated with `discount_damaged_good`	High correlation
`Store` is highly correlated with `CPI` and 1 other fields	High correlation
`gas_price` is highly correlated with `Unemployment`	High correlation
`CPI` is highly correlated with `Store` and 1 other fields	High correlation
`Unemployment` is highly correlated with `Store` and 2 other fields	High correlation
`discount_promotional` has 3146 (49.4%) missing values	Missing
`discount_clearance` has 3870 (60.8%) missing values	Missing
`discount_damaged_good` has 3400 (53.4%) missing values	Missing
`discount_competitive` has 3305 (51.9%) missing values	Missing
`CPI` has 468 (7.3%) missing values	Missing
`Unemployment` has 455 (7.1%) missing values	Missing
`Date` is uniformly distributed	Uniform

Reproduction

Analysis started	2022-11-05 05:47:28.998892
Analysis finished	2022-11-05 05:47:36.964294
Duration	7.97 seconds
Software version	pandas-profiling v3.4.0
Download configuration	config.json

Store
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	35
Distinct (%)	0.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	18

Minimum	1
Maximum	35
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	49.9 KiB

Quantile statistics

Minimum	1
5-th percentile	2
Q1	9
median	18
Q3	27
95-th percentile	34
Maximum	35
Range	34
Interquartile range (IQR)	18

Descriptive statistics

Standard deviation	10.10029777
Coefficient of variation (CV)	0.561127654
Kurtosis	-1.201962265
Mean	18
Median Absolute Deviation (MAD)	9
Skewness	0
Sum	114660
Variance	102.0160151
Monotonicity	Increasing

Histogram with fixed size bins (bins=35)

Value	Count	Frequency (%)
1	182	2.9%
27	182	2.9%
21	182	2.9%
22	182	2.9%
23	182	2.9%
24	182	2.9%
25	182	2.9%
26	182	2.9%
28	182	2.9%
19	182	2.9%
Other values (25)	4550	71.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	182	2.9%
2	182	2.9%
3	182	2.9%
4	182	2.9%
5	182	2.9%
6	182	2.9%
7	182	2.9%
8	182	2.9%
9	182	2.9%
10	182	2.9%

Value	Count	Frequency (%)
35	182	2.9%
34	182	2.9%
33	182	2.9%
32	182	2.9%
31	182	2.9%
30	182	2.9%
29	182	2.9%
28	182	2.9%
27	182	2.9%
26	182	2.9%

Date
Categorical

HIGH CARDINALITY
UNIFORM

Distinct	182
Distinct (%)	2.9%
Missing	0
Missing (%)	0.0%
Memory size	49.9 KiB

2/5/2010	35
4/13/2012	35
4/27/2012	35
5/4/2012	35
5/11/2012	35
Other values (177)	6195

Length

Max length	10
Median length	9
Mean length	8.923076923
Min length	8

Characters and Unicode

Total characters	56840
Distinct characters	11
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	2/5/2010
2nd row	2/12/2010
3rd row	2/19/2010
4th row	2/26/2010
5th row	3/5/2010

Common Values

Value	Count	Frequency (%)
2/5/2010	35	0.5%
4/13/2012	35	0.5%
4/27/2012	35	0.5%
5/4/2012	35	0.5%
5/11/2012	35	0.5%
5/18/2012	35	0.5%
5/25/2012	35	0.5%
6/1/2012	35	0.5%
6/8/2012	35	0.5%
6/15/2012	35	0.5%
Other values (172)	6020	94.5%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
2/5/2010	35	0.5%
7/9/2010	35	0.5%
7/2/2010	35	0.5%
2/19/2010	35	0.5%
2/26/2010	35	0.5%
3/5/2010	35	0.5%
3/12/2010	35	0.5%
3/19/2010	35	0.5%
3/26/2010	35	0.5%
4/2/2010	35	0.5%
Other values (172)	6020	94.5%

Most occurring characters

Value	Count	Frequency (%)
1	13335	23.5%
/	12740	22.4%
2	11935	21.0%
0	9100	16.0%
3	2590	4.6%
5	1260	2.2%
4	1260	2.2%
7	1225	2.2%
6	1225	2.2%
8	1085	1.9%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	44100	77.6%
Other Punctuation	12740	22.4%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
1	13335	30.2%
2	11935	27.1%
0	9100	20.6%
3	2590	5.9%
5	1260	2.9%
4	1260	2.9%
7	1225	2.8%
6	1225	2.8%
8	1085	2.5%
9	1085	2.5%

Other Punctuation

Value	Count	Frequency (%)
/	12740	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	56840	100.0%

Most frequent character per script

Common

Value	Count	Frequency (%)
1	13335	23.5%
/	12740	22.4%
2	11935	21.0%
0	9100	16.0%
3	2590	4.6%
5	1260	2.2%
4	1260	2.2%
7	1225	2.2%
6	1225	2.2%
8	1085	1.9%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	56840	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
1	13335	23.5%
/	12740	22.4%
2	11935	21.0%
0	9100	16.0%
3	2590	4.6%
5	1260	2.2%
4	1260	2.2%
7	1225	2.2%
6	1225	2.2%
8	1085	1.9%

IsHoliday
Boolean

HIGH CORRELATION

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	6.3 KiB

False	5915
True	455

Common Values
Category Frequency Plot

Value	Count	Frequency (%)
False	5915	92.9%
True	455	7.1%

Temperature
Real number (ℝ)

Distinct	3759
Distinct (%)	59.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	58.60789011

Minimum	-7.29
Maximum	101.95
Zeros	0
Zeros (%)	0.0%
Negative	4
Negative (%)	0.1%
Memory size	49.9 KiB

Quantile statistics

Minimum	-7.29
5-th percentile	26.49
Q1	45.015
median	59.91
Q3	72.99
95-th percentile	87.0155
Maximum	101.95
Range	109.24
Interquartile range (IQR)	27.975

Descriptive statistics

Standard deviation	18.72455401
Coefficient of variation (CV)	0.3194886214
Kurtosis	-0.6143786632
Mean	58.60789011
Median Absolute Deviation (MAD)	13.96
Skewness	-0.2473546907
Sum	373332.26
Variance	350.608923
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
70.28	9	0.1%
67.87	9	0.1%
50.43	9	0.1%
46.28	8	0.1%
47.55	7	0.1%
64.21	7	0.1%
64.28	7	0.1%
59.94	7	0.1%
62.62	6	0.1%
45	6	0.1%
Other values (3749)	6295	98.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-7.29	1	< 0.1%
-6.61	1	< 0.1%
-6.08	1	< 0.1%
-2.06	1	< 0.1%
0.25	1	< 0.1%
2.32	1	< 0.1%
2.45	1	< 0.1%
4	1	< 0.1%
5.54	1	< 0.1%
6.23	1	< 0.1%

Value	Count	Frequency (%)
101.95	2	< 0.1%
100.14	1	< 0.1%
100.07	1	< 0.1%
99.66	2	< 0.1%
99.22	2	< 0.1%
99.2	1	< 0.1%
98.43	1	< 0.1%
98.15	1	< 0.1%
97.66	1	< 0.1%
97.6	1	< 0.1%

gas_price
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	943
Distinct (%)	14.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3.41654898

Minimum	2.514
Maximum	4.468
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	49.9 KiB

Quantile statistics

Minimum	2.514
5-th percentile	2.689
Q1	3.054
median	3.524
Q3	3.749
95-th percentile	4.0241
Maximum	4.468
Range	1.954
Interquartile range (IQR)	0.695

Descriptive statistics

Standard deviation	0.4278726182
Coefficient of variation (CV)	0.1252353239
Kurtosis	-0.9312876775
Mean	3.41654898
Median Absolute Deviation (MAD)	0.292
Skewness	-0.329129282
Sum	21763.417
Variance	0.1830749774
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3.417	36	0.6%
3.638	34	0.5%
3.63	33	0.5%
3.583	32	0.5%
3.62	32	0.5%
3.227	26	0.4%
3.666	26	0.4%
3.611	25	0.4%
3.622	24	0.4%
3.386	24	0.4%
Other values (933)	6078	95.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
2.514	10	0.2%
2.54	3	< 0.1%
2.548	10	0.2%
2.55	3	< 0.1%
2.561	10	0.2%
2.565	10	0.2%
2.572	13	0.2%
2.573	3	< 0.1%
2.574	2	< 0.1%
2.577	10	0.2%

Value	Count	Frequency (%)
4.468	4	0.1%
4.449	4	0.1%
4.308	2	< 0.1%
4.301	4	0.1%
4.294	4	0.1%
4.293	2	< 0.1%
4.288	2	< 0.1%
4.282	2	< 0.1%
4.277	4	0.1%
4.273	4	0.1%

discount_promotional
Real number (ℝ)

HIGH CORRELATION
MISSING

Distinct	3141
Distinct (%)	97.4%
Missing	3146
Missing (%)	49.4%
Infinite	0
Infinite (%)	0.0%
Mean	7832.565958

Minimum	-2781.45
Maximum	103184.98
Zeros	0
Zeros (%)	0.0%
Negative	2
Negative (%)	< 0.1%
Memory size	49.9 KiB

Quantile statistics

Minimum	-2781.45
5-th percentile	370.99
Q1	2638.645
median	5427.955
Q3	9553.4425
95-th percentile	22692.948
Maximum	103184.98
Range	105966.43
Interquartile range (IQR)	6914.7975

Descriptive statistics

Standard deviation	9512.725241
Coefficient of variation (CV)	1.214509433
Kurtosis	23.87362661
Mean	7832.565958
Median Absolute Deviation (MAD)	3198.88
Skewness	4.079904893
Sum	25252192.65
Variance	90491941.52
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9667.5	2	< 0.1%
29091.04	2	< 0.1%
9649.15	2	< 0.1%
2177.83	2	< 0.1%
2657.14	2	< 0.1%
3634.62	2	< 0.1%
6458.44	2	< 0.1%
3805.93	2	< 0.1%
2005.31	2	< 0.1%
7962.18	2	< 0.1%
Other values (3131)	3204	50.3%
(Missing)	3146	49.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-2781.45	1	< 0.1%
-563.9	1	< 0.1%
0.5	1	< 0.1%
1.5	1	< 0.1%
2.5	1	< 0.1%
2.82	1	< 0.1%
5.76	1	< 0.1%
7.82	1	< 0.1%
8.62	1	< 0.1%
9.59	1	< 0.1%

Value	Count	Frequency (%)
103184.98	1	< 0.1%
95102.5	1	< 0.1%
88750.34	1	< 0.1%
88646.76	1	< 0.1%
84139.36	1	< 0.1%
80498.65	1	< 0.1%
78124.5	1	< 0.1%
77017.24	1	< 0.1%
75522.86	1	< 0.1%
75149.79	1	< 0.1%

discount_clearance
Real number (ℝ)

MISSING

Distinct	2282
Distinct (%)	91.3%
Missing	3870
Missing (%)	60.8%
Infinite	0
Infinite (%)	0.0%
Mean	3536.943988

Minimum	-265.76
Maximum	104519.54
Zeros	3
Zeros (%)	< 0.1%
Negative	23
Negative (%)	0.4%
Memory size	49.9 KiB

Quantile statistics

Minimum	-265.76
5-th percentile	3.195
Q1	70.74
median	360.28
Q3	2413.8225
95-th percentile	18657.7065
Maximum	104519.54
Range	104785.3
Interquartile range (IQR)	2343.0825

Descriptive statistics

Standard deviation	9108.132594
Coefficient of variation (CV)	2.575141881
Kurtosis	31.31713162
Mean	3536.943988
Median Absolute Deviation (MAD)	350.79
Skewness	4.895291162
Sum	8842359.97
Variance	82958079.35
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1.91	9	0.1%
4	7	0.1%
3	7	0.1%
19	6	0.1%
0.03	6	0.1%
0.01	6	0.1%
11	5	0.1%
9	5	0.1%
5.73	5	0.1%
3.82	5	0.1%
Other values (2272)	2439	38.3%
(Missing)	3870	60.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-265.76	1	< 0.1%
-192	1	< 0.1%
-35.74	1	< 0.1%
-15.45	1	< 0.1%
-10.98	1	< 0.1%
-10.5	2	< 0.1%
-9.98	1	< 0.1%
-9.94	1	< 0.1%
-7.76	1	< 0.1%
-7.6	1	< 0.1%

Value	Count	Frequency (%)
104519.54	1	< 0.1%
97740.99	1	< 0.1%
92523.94	1	< 0.1%
89121.94	1	< 0.1%
82881.16	1	< 0.1%
72413.71	1	< 0.1%
71074.17	1	< 0.1%
70574.85	1	< 0.1%
59362.3	1	< 0.1%
58804.91	1	< 0.1%

discount_damaged_good
Real number (ℝ)

HIGH CORRELATION
MISSING

Distinct	2437
Distinct (%)	82.1%
Missing	3400
Missing (%)	53.4%
Infinite	0
Infinite (%)	0.0%
Mean	1946.11564

Minimum	-179.26
Maximum	149483.31
Zeros	1
Zeros (%)	< 0.1%
Negative	9
Negative (%)	0.1%
Memory size	49.9 KiB

Quantile statistics

Minimum	-179.26
5-th percentile	0.8
Q1	7.7
median	44.99
Q3	185.2825
95-th percentile	1408.6095
Maximum	149483.31
Range	149662.57
Interquartile range (IQR)	177.5825

Descriptive statistics

Standard deviation	11850.71185
Coefficient of variation (CV)	6.089418127
Kurtosis	65.89481834
Mean	1946.11564
Median Absolute Deviation (MAD)	42.59
Skewness	7.765311128
Sum	5779963.45
Variance	140439371.4
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1	15	0.2%
3	14	0.2%
2	11	0.2%
4	10	0.2%
0.6	10	0.2%
6	9	0.1%
0.5	8	0.1%
0.3	8	0.1%
1.2	8	0.1%
0.22	7	0.1%
Other values (2427)	2870	45.1%
(Missing)	3400	53.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-179.26	1	< 0.1%
-89.1	1	< 0.1%
-44.54	1	< 0.1%
-29.1	1	< 0.1%
-23.97	1	< 0.1%
-14.29	1	< 0.1%
-2.58	1	< 0.1%
-1	1	< 0.1%
-0.86	1	< 0.1%
0	1	< 0.1%

Value	Count	Frequency (%)
149483.31	1	< 0.1%
146394.44	1	< 0.1%
141630.61	1	< 0.1%
139621.51	1	< 0.1%
130129.11	1	< 0.1%
115048.81	1	< 0.1%
112255.67	1	< 0.1%
109030.75	1	< 0.1%
105691.67	1	< 0.1%
105146.3	1	< 0.1%

discount_competitive
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	2959
Distinct (%)	96.5%
Missing	3305
Missing (%)	51.9%
Infinite	0
Infinite (%)	0.0%
Mean	3330.461077

Minimum	0.46
Maximum	67474.85
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	49.9 KiB

Quantile statistics

Minimum	0.46
5-th percentile	27.022
Q1	354.12
median	1201.3
Q3	3338.11
95-th percentile	12861.768
Maximum	67474.85
Range	67474.39
Interquartile range (IQR)	2983.99

Descriptive statistics

Standard deviation	6824.986916
Coefficient of variation (CV)	2.049261877
Kurtosis	28.98549732
Mean	3330.461077
Median Absolute Deviation (MAD)	1060.72
Skewness	4.878394693
Sum	10207863.2
Variance	46580446.4
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9	4	0.1%
3	3	< 0.1%
8	3	< 0.1%
813.77	3	< 0.1%
2.5	3	< 0.1%
149.77	2	< 0.1%
496.24	2	< 0.1%
262.81	2	< 0.1%
433.38	2	< 0.1%
1534.4	2	< 0.1%
Other values (2949)	3039	47.7%
(Missing)	3305	51.9%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0.46	1	< 0.1%
0.63	1	< 0.1%
0.66	1	< 0.1%
0.87	1	< 0.1%
1.88	1	< 0.1%
1.92	1	< 0.1%
1.94	1	< 0.1%
2	2	< 0.1%
2.5	3	< 0.1%
2.52	1	< 0.1%

Value	Count	Frequency (%)
67474.85	1	< 0.1%
65344.64	1	< 0.1%
63130.81	1	< 0.1%
60065.82	1	< 0.1%
57817.56	1	< 0.1%
57815.43	1	< 0.1%
56735.25	1	< 0.1%
56600.97	1	< 0.1%
53603.99	1	< 0.1%
52850.8	1	< 0.1%

CPI
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	2089
Distinct (%)	35.4%
Missing	468
Missing (%)	7.3%
Infinite	0
Infinite (%)	0.0%
Mean	171.3886234

Minimum	126.064
Maximum	228.9764563
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	49.9 KiB

Quantile statistics

Minimum	126.064
5-th percentile	126.6019032
Q1	132.5461333
median	142.4460738
Q3	214.7027646
95-th percentile	225.1732488
Maximum	228.9764563
Range	102.9124563
Interquartile range (IQR)	82.1566313

Descriptive statistics

Standard deviation	40.24166294
Coefficient of variation (CV)	0.2347977488
Kurtosis	-1.822368068
Mean	171.3886234
Median Absolute Deviation (MAD)	16.3100093
Skewness	0.1627910466
Sum	1011535.655
Variance	1619.391436
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
132.7160968	24	0.4%
139.1226129	21	0.3%
224.8025314	12	0.2%
143.2200737	9	0.1%
228.7298638	9	0.1%
126.5522857	9	0.1%
126.4962581	9	0.1%
126.4420645	9	0.1%
126.5262857	9	0.1%
201.0705712	9	0.1%
Other values (2079)	5782	90.8%
(Missing)	468	7.3%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
126.064	8	0.1%
126.0766452	8	0.1%
126.0854516	8	0.1%
126.0892903	8	0.1%
126.1019355	8	0.1%
126.1069032	8	0.1%
126.1119032	8	0.1%
126.114	8	0.1%
126.1145806	8	0.1%
126.1266	8	0.1%

Value	Count	Frequency (%)
228.9764563	3	< 0.1%
228.8892482	1	< 0.1%
228.8020401	1	< 0.1%
228.7796682	3	< 0.1%
228.7298638	9	0.1%
228.714832	1	< 0.1%
228.6926456	1	< 0.1%
228.6428882	3	< 0.1%
228.6276239	1	< 0.1%
228.605623	1	< 0.1%

Unemployment
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	352
Distinct (%)	6.0%
Missing	455
Missing (%)	7.1%
Infinite	0
Infinite (%)	0.0%
Mean	7.759504818

Minimum	3.684
Maximum	14.313
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	49.9 KiB

Quantile statistics

Minimum	3.684
5-th percentile	5.277
Q1	6.617
median	7.767
Q3	8.513
95-th percentile	10.581
Maximum	14.313
Range	10.629
Interquartile range (IQR)	1.896

Descriptive statistics

Standard deviation	1.7808631
Coefficient of variation (CV)	0.2295073128
Kurtosis	3.012236015
Mean	7.759504818
Median Absolute Deviation (MAD)	0.9
Skewness	1.15741994
Sum	45897.471
Variance	3.17147338
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
8.099	78	1.2%
7.852	70	1.1%
7.931	65	1.0%
6.891	65	1.0%
7.057	65	1.0%
7.441	65	1.0%
6.565	65	1.0%
6.17	65	1.0%
6.237	60	0.9%
8.028	57	0.9%
Other values (342)	5260	82.6%
(Missing)	455	7.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
3.684	4	0.1%
3.879	13	0.2%
3.896	4	0.1%
3.921	13	0.2%
3.932	13	0.2%
4.077	13	0.2%
4.125	13	0.2%
4.145	13	0.2%
4.156	13	0.2%
4.261	13	0.2%

Value	Count	Frequency (%)
14.313	28	0.4%
14.18	26	0.4%
14.099	26	0.4%
14.021	24	0.4%
13.975	16	0.3%
13.736	26	0.4%
13.503	28	0.4%
12.89	26	0.4%
12.187	26	0.4%
11.627	26	0.4%

Auto

The auto setting is an easily interpretable pairwise column metric of the following mapping: vartype-vartype : method, categorical-categorical : Cramer's V, numerical-categorical : Cramer's V (using a discretized numerical column), numerical-numerical : Spearman's ρ. This configuration uses the best suitable for each pair of columns.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The correlation heatmap measures nullity correlation: how strongly the presence or absence of one variable affects the presence of another.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	Store	Date	IsHoliday	Temperature	gas_price	discount_promotional	discount_clearance	discount_damaged_good	discount_competitive	CPI	Unemployment
0	1	2/5/2010	False	59.33	3.360	9667.50	268.29	0.60	8368.15	223.659114	6.833
1	1	2/12/2010	True	51.65	3.409	8687.47	1594.87	2.20	2144.87	223.753643	6.833
2	1	2/19/2010	False	52.39	3.510	2706.87	3128.74	1.88	2396.68	223.917015	6.833
3	1	2/26/2010	False	60.12	3.555	6129.28	1802.84	NaN	301.48	224.132020	6.833
4	1	3/5/2010	False	61.65	3.630	3552.58	601.32	NaN	2666.22	224.347025	6.833
5	1	3/12/2010	False	60.71	3.669	8368.50	298.46	1.39	1340.29	224.562029	6.833
6	1	3/19/2010	False	64.00	3.734	2588.06	80.89	3.20	909.76	224.716695	6.833
7	1	3/26/2010	False	66.53	3.787	737.08	36.10	1.43	92.11	224.790910	6.833
8	1	4/2/2010	False	69.36	3.845	825.10	99.83	1.25	113.25	224.865125	6.833
9	1	4/9/2010	False	73.01	3.891	4433.71	NaN	24.45	1282.30	224.939340	6.664

Last rows

	Store	Date	IsHoliday	Temperature	gas_price	discount_promotional	discount_clearance	discount_damaged_good	discount_competitive	CPI	Unemployment
6360	35	5/24/2013	False	65.19	3.627	3116.05	598.90	471.12	1030.77	NaN	NaN
6361	35	5/31/2013	False	64.65	3.646	4542.19	286.29	13.80	36.90	NaN	NaN
6362	35	6/7/2013	False	69.53	3.633	14769.81	154.12	53.59	6333.27	NaN	NaN
6363	35	6/14/2013	False	67.64	3.632	6636.89	159.07	313.37	2895.53	NaN	NaN
6364	35	6/21/2013	False	70.76	3.626	3252.39	118.02	65.02	4722.43	NaN	NaN
6365	35	6/28/2013	False	77.34	3.639	4764.55	85.65	NaN	4660.01	NaN	NaN
6366	35	7/5/2013	False	77.41	3.614	6333.87	138.09	610.35	7224.45	NaN	NaN
6367	35	7/12/2013	False	80.74	3.614	4798.18	130.88	22.05	2351.60	NaN	NaN
6368	35	7/19/2013	False	83.36	3.737	2318.53	122.30	23.45	771.35	NaN	NaN
6369	35	7/26/2013	False	77.01	3.804	237.96	50.00	4.00	23.64	NaN	NaN

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Other Punctuation

Most occurring scripts

Most frequent character per script

Common

Most occurring blocks

Most frequent character per block

ASCII

Interactions

Correlations

Auto

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows